Table prioritization for data copy in a multi-environment setup

ABSTRACT

A system includes one or more source memory devices of a source computing environment that store a database comprising data files, wherein each of a plurality of data tables of the source computing environment includes data from one or more of the data files, one or more target memory devices of a target computing environment and at least one processor configured to receive a command to copy data files from the source memory devices to the target memory devices, detect that the target memory devices have insufficient memory, calculate a value coefficient for each data table, assign a priority index to each data table based on the value coefficient, order the data files in a copy queue based on the priority index of the data tables, and copy the ordered data files to the target memory devices.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/656,060 filed Mar. 23, 2022, and entitled “TABLE PRIORITIZATION FORDATA COPY IN A MULTI-ENVIRONMENT SETUP,” which is incorporated herein byreference.

TECHNICAL FIELD

The present disclosure relates generally to data processing, and morespecifically to table prioritization for data copy in amulti-environment setup.

BACKGROUND

In a multi-environment system, a target computing environment oftenundergoes periodic restoration which includes copying data stored in oneor more source memory devices of a source computing environment to oneor more target memory devices of the target computing environment. Eachrestoration event essentially creates an image of the source computingenvironment in the target computing environment. A common errorassociated with copying data between computing environments (e.g.,source computing environment to target computing environment) in amulti-environment system is a restoration failure or abort as a resultof the target environment having insufficient memory space for copyingall data from the source environment. In most such failure events, evenif the target memory devices have sufficient memory space to accommodateall critical data files (e.g., critical data tables) needed foroperating the target environment, a data restoration may still fail aspresently no mechanism exists to selectively copy the critical datafiles first before the target memory devices run out of memory space.

SUMMARY

The system and methods implemented by the system as disclosed in thepresent disclosure provide techniques for automatically andintelligently prioritizing data tables of a database and copying datarelating to higher priority data tables before copying data relating tolower priority data tables. The disclosed system and methods provideseveral practical applications and technical advantages.

For example, the disclosed system and methods provide the practicalapplication of automatically and intelligently prioritizing data tablesand file groups of a database at a source computing environment based onpre-defined metrics so that data files associated with critical datatables are prioritized during a data copy to a target computingenvironment. As described in accordance with embodiments of the presentdisclosure a copy manager determines a value coefficient for each datatable of a database based at least on table metadata and user-definedmetrics related to the data table. A priority index is assigned to eachdata table based on the value coefficient of the data table, wherein ahigher priority index is assigned to a data table having a higher valuecoefficient. The copy manager schedules copying of file groups and datafiles in order of the priority indices assigned to respective datatables starting with data files containing data relating to data tableswith the highest assigned priority indices. For example, copy managerre-arranges the file groups such that data files containing datarelating to data files with higher priority indices are re-assigned tofile groups scheduled to be copied earlier to the target memory devices,and data files containing data relating to data files with lowerpriority indices are assigned to file groups scheduled to be copiedlater to the target memory devices.

By identifying and copying critical data tables first, the describedsystem and methods may ensure that critical data needed for optimallyoperating the target computing environment is copied to the targetcomputing environment before the target memory devices run out of memoryspace. Thus, even when the target memory devices have insufficientmemory space to store all data from the source memory devices, a datarestoration may not fail and the target computing environment mayoperate optimally as the critical data tables and associated data filesmay have already been copied to the target memory devices when thetarget memory devices run out of memory space during a data copy. Thus,the disclosed system and methods improve the technology related to datarestoration between computing environments in a multi-environmentsystem.

The disclosed system and methods provide an additional technicaladvantage of improving performance of a computing system configured torun computing environments or portions thereof in a multi-environmentsystem. For example, in the event that a target computing environmenthas insufficient memory space to store all data files from a sourcecomputing environment, the disclosed system and methods may help ensurethat most critical data is copied to the target computing environment sothat a data restoration operation does not fail and the target computingenvironment operates optimally after each data restore. By avoiding datacopy failures and errors in the operation of the target computingenvironment as a result of critical data files not being copied, thedisclosed system and methods improve the efficiency of the targetcomputing environment and the overall efficiently of a multi-environmentsystem. This in turn improves the processing performance of thecomputing system running the computing environments.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure, reference is nowmade to the following brief description, taken in connection with theaccompanying drawings and detailed description, wherein like referencenumerals represent like parts.

FIG. 1 is a schematic diagram of an example data processing system, inaccordance with one or more embodiments of the present disclosure;

FIG. 2 illustrates an example calculation of criticality index for datatables based on data lineage information of data tables, in accordancewith one or more embodiments of the present disclosure;

FIGS. 3A and 3B illustrate an example calculation of value coefficientsfor data tables, in accordance with one or more embodiments of thepresent disclosure;

FIG. 4 is a flowchart of an example method for managing a data copy froma source computing environment to a target computing environment, inaccordance with one or more embodiments of the present disclosure;

FIG. 5 is a flowchart of an example method for valuating a data table,in accordance with one or more embodiments of the present disclosure;and

FIG. 6 illustrates an example schematic diagram of the copy managerillustrated in FIG. 1 , in accordance with one or more embodiments ofthe present disclosure.

DETAILED DESCRIPTION System Overview

FIG. 1 is a schematic diagram of an example data processing system 100,in accordance with one or more embodiments of the present disclosure.

As shown in FIG. 1 , data processing system 100 may include sourcecomputing environment 150, target computing environment 170, copymanager 110 and user devices 190, each connected to network 180. Network180, in general, may be a wide area network (WAN), a personal areanetwork (PAN), a cellular network, or any other technology that allowsdevices to communicate electronically with other devices. In one or moreembodiments, network 180 may be the Internet. Each user device 190 maybe a computing device that can be operated by a user 192 and communicatewith other devices connected to the network 180.

In one or more embodiments, each of the source computing environment150, target computing environment 170, copy manager 110 and user devices190 may be representative of a computing system hosting softwareapplications that may be installed and run locally or may be used toaccess software applications running on a server (not shown). Thecomputing system may include mobile computing systems including smartphones, tablet computers, laptop computers, or any other mobilecomputing devices or systems capable of running software applicationsand communicating with other devices. The computing system may alsoinclude non-mobile computing devices such as desktop computers or othernon-mobile computing devices capable of running software applicationsand communicating with other devices. In certain embodiments, one ormore of the source computing environment 150, target computingenvironment 170, copy manager 110 and user devices 190 may berepresentative of a server running one or more software applications toimplement respective functionality as described below. In certainembodiments, one or more of the source computing environment 150, targetcomputing environment 170, copy manager 110 and user devices 190 may runa thin client software application where the processing is directed bythe thin client but largely performed by a central entity such as aserver (not shown).

Each of the source computing environment 150 and the target computingenvironment 170 may represent a computing environment of anorganization. For example, the source computing environment 150 mayrepresent a production computing environment where the latest versionsof software, products or updates are pushed live to the intended users.A production computing environment generally can be thought of as areal-time computing system where computer programs are run and hardwaresetups are installed and relied on for an organization's dailyoperations. In one embodiment, the target computing environment 170 mayrepresent a lower level environment such as a development environment ortesting environment. A development environment in software and webdevelopment generally refers to a workspace for software developers tomake changes to one or more software applications without affecting alive environment such as a production environment. A test environmentgenerally refers to a workspace a series of tests can be conducted on asoftware application before deployment in a production environment.

As shown, each of the source computing environment 150 and the targetcomputing environment 170 may include a plurality of components 152 and172 respectively, including one or more hardware devices and one or moresoftware applications. Each component 152 or 172 may include a hardwaredevice or a software application. Hardware devices may include, but arenot limited to, one or more processors, one or more memory devices,servers, desktop computer, mobile computing devices, printed circuitboards (e.g., display cards, sound cards, interface cards etc.),electronic components (e.g. transistors, diodes, capacitors, resistorsetc.) and machines. Software applications may include software programsincluding, but not limited to, operating systems, user interfaceapplications, third party software, database management software andother customized software programs implementing particularfunctionalities in each of the computing environments 150 and 170. In anembodiment, one or more software applications are run using hardwaredevices to implement one or more functionalities in a computingenvironment 150 or 170. For example, software code relating to one ormore software applications may be stored in a memory device and one ormore processors may process the software code to implement respectivefunctionalities in the computing environment 150 and 170.

It may be noted that while FIG. 1 shows the system 100 as including twocomputing environments, a person having ordinary skill in the art mayappreciate that system 100 may include more than two computingenvironments.

As shown in FIG. 1 , source computing environment 150 may have one ormore source memory devices 154 that store data for the source computingenvironment 150. The memory devices 154 may include one or more primarystorage devices such as Random-Access Memory (RAM) or one or moresecondary storage devices including but not limited to magnetic disks,optical disks, hard disks, flash disks and magnetic tapes. For example,as shown in FIG. 1 , source memory device 154 may store a database 156(shown as DB1). Data relating to the database 156 is physically storedin the source memory devices 154 as a plurality of data files 158grouped into a plurality of file groups 159 (shown as FG1, FG2 . . .FGN). Data files 158 of the database 156 may include one or more primarydata files, one or more secondary data files and one or more log datafiles. The primary and secondary data files may contain data and objectssuch as data tables, indexes, stored procedures and views. A log datafile may hold transaction log information relating to the database 156.In one embodiment, database 156 includes a primary data file, one ormore secondary data files which are typically user-defined data files,and one or more log data files. For example, a simple database mayinclude one primary data file that contains all data and objects and alog data file that contains the transaction log information. A morecomplex database such as a database storing employee information for alarge organization may include one primary data file, several secondarydata files and several log files. The data and objects within thedatabase may be spread across the primary data file and the severalsecondary data files, and all the log files may include the transactionlog information for the database. Database 156 may include several filegroups 159 (shown as FG1, FG2 . . . FGN). File groups 159 may include aprimary file group and one or more user-defined file groups. The primaryfile group generally includes the primary data file and any secondarydata files that are not in other file groups. A file group 159 mayinclude data files 158 stored in separate storage device (e.g., diskdrives). For example, data files Data1, Data 2 and Data 3 may be storedon three separate disk drives and assigned to a single file group FG1.One or more data tables 160 may be created based on data stored in oneor more data files 158 of one or more file groups 159. For example, adata table 160 may be created that includes data from each of the datafiles Data1, Data 2 and Data 3 (spread across three separate diskdrives) in file group FG1. In this case, queries for data from the datatable 160 is spread across the three disks.

As shown in FIG. 1 , the target computing environment 170 may have oneor more target memory devices 174 that store data for the targetcomputing environment 170. The memory devices 174 may include one ormore primary storage devices such as Random-Access Memory (RAM) or oneor more secondary storage devices including but not limited to magneticdisks, optical disks, hard disks, flash disks and magnetic tapes.

In some cases, the target computing environment 170 undergoes periodicrestoration which includes copying data stored in the source memorydevices 154 to the target memory devices 174. Each restoration eventessentially creates an image of the source computing environment 150 inthe target computing environment 170. For example, the source computingenvironment 150 may be a production environment of an organization andthe target computing environment may be a lower level environment suchas a development environment or test environment. Software developersmay create and test software patches or updates for one or more softwareapplications in the image of the production environment stored in thelower level environment so that there is no service interruption in theproduction environment. Once ready, the software patch or update may beapplied to the respective software application in the live productionenvironment. However, in some cases, the target computing environment170 may have insufficient storage capacity within the target memorydevices 174 to accommodate the entire data stored in the source memorydevices 154 during a data copy. A common error associated with copyingdata between computing environments (e.g., source computing environment150 to target computing environment 170) in a multi-environment system(such as system 100) is a restoration failure or abort as a result ofthe target environment having insufficient memory space for copying alldata from the source environment. In most such failure events, even ifthe target memory devices have sufficient memory space to accommodateall critical data files (e.g., critical data tables) needed foroperating the target environment, a data restoration may still fail aspresently no mechanism exists to select and copy the critical data filesfirst so that only non-critical data files and tables are left uncopiedwhen the target memory devices run out of memory space.

Embodiments of the present disclosure describe techniques forautomatically and intelligently prioritizing data tables 160 and filegroups 159 at a source computing environment 150 based on pre-definedmetrics so that data files 158 associated with critical data tables 160are prioritized during a data copy to a target computing environment170. By identifying and copying critical data tables 160 first, thedescribed system and methods may ensure that critical data needed foroptimally operating the target computing environment 170 is copied tothe target computing environment 170 before the target memory devices174 run out of memory space. Thus, even when the target memory devices174 have insufficient memory space to store all data from the sourcememory devices 154, a data restoration may not fail and the targetcomputing environment 170 may operate optimally or near optimally as thecritical data tables 160 and associated data files 158 may have beencopied to the target memory devices 174. It may be noted that the terms“data restore or restoration” and “data copy” are used interchangeablythroughout this disclosure.

Copy manager 110 may be configured to manage a data copy or datarestoration from the source computing environment 150 to the targetcomputing environment 170. Copy manager 110 may receive a request for adata restore or data copy from the source computing environment 150 tothe target computing environment 170. The request for the data copy maybe generated by a user 192 using a user device 190. The user 192generating the request may be an administrator of the source computingenvironment 150 and/or the target computing environment 170 or any otheruser 192 with appropriate credentials to make such a request. Inresponse to receiving the request for data copy, copy manager 110 maydetermine whether the target memory devices 174 have sufficient memoryspace to receive and store the entire database 156 (or otherwise alldata) stored in the source memory devices 154. Copy manager 110 may haveaccess to environment metadata 134 including metadata relating to thesource computing environment 150 and the target computing environment170. The environment metadata 134 may include information relating to asize of each of the source memory devices 154, a size of each of thetarget memory devices 174, a size of each data table 160 of the database156, and memory space needed in each of the target memory devices 174for operational purposes. Copy manager 110 may be configured tocalculate a total amount of data to be copied from the source memorydevices 154 to the target memory devices 174 based on the individualdata table sizes of the data tables 160. For example, copy manager 110may add the sizes of all data tables 160 (e.g., obtained from theenvironment metadata 134) to determine the total data size to be copiedto the target memory devices 174. Copy manager 110 may be configured todetermine a combined memory space available at the target memory devices174 (e.g., when the target memory devices include multiple storagedisks) to store data received from the source memory devices 154. Forexample, copy manager 110 may calculate the available memory at thetarget memory devices 174 by subtracting the total memory space neededat the target memory devices 174 for operational purposes from the totalcombined memory space of target memory devices 174. Copy manager 110 maybe configured to compare the total data size to be copied from thesource memory devices 154 with the total memory space available at thetarget memory devices 174 to store the data. When the memory spaceavailable at the target memory devices 174 is less than the total datasize to be copied from the source memory devices 154, copy manager maybe configured to determine that the target memory devices 174 haveinsufficient memory space to receive and store the entire database 156(or otherwise all data) stored in the source memory devices 154.

In response to determining that target memory devices 174 haveinsufficient memory space to store the entire database 156 (or otherwiseall data) stored in the source memory devices 154, copy manager 110 maybe configured to assign priorities (e.g., priority index 149 as shown inFIG. 1 ) to data tables 160 based on pre-defined metrics, and copy datafiles 158 and file groups 159 relating to the data tables 160 in theorder of respective priorities assigned to the data tables 160 startingwith data files 158 having data for data tables 160 with the highestassigned priorities. As further described below, copy manager 110 mayassign priorities to each data table 160 of the database 156 based on arelative importance of the data table 160 (as determined based on thepre-defined metrics) among a plurality of data tables 160 defined forthe database 156. This technique may ensure that data relating to themost critical data tables 160 is copied prior to data relatingrelatively non-critical data tables 160. As further described below, oneor more of the pre-defined metrics may be indicative of or can be usedto determine the importance of each data table 160 in the context of thetarget computing environment 170. Each pre-defined metric associatedwith a data table 160 may have a default value based on the nature ofdata contained in the data table 160 or a user-defined value to suitparticular needs of the target computing environment 170.

Copy manager 110 may have access to a plurality of pre-defined metricsrelating to each data table 160. As shown in FIG. 1 , the pre-definedmetrics may include, but are not limited to, table metadata 112,user-defined metrics 122 and environment metadata 134. Table metadata112 may include, but is limited to, table interaction statistics 114,table size 115, importance index 116, data lineage repository 118 andquery execution history 120. Table interaction statistics 114 relatingto a data table 160 may include information regarding an amount of oneor more types of interactions performed in relation to the data table160. For example, table interaction statistics 114 may include a totalnumber of read operations performed in the data table 160 and a totalnumber of write operations performed in the data table 160. Table size115 may include a size of a data table 160 indicative of a total memoryspace occupied by the data table 160 in the source memory devices 154.Importance index 116 relating to a data table 160 may be indicative ofan importance of the data table 160 in the source computing environment150. Importance index 116 may include Key Performance Indicator (KPI)index typically assigned to the data table 160 based on how importantthe data table is for the source computing environment 150 and/or thetarget computing environment 170. The terms “importance index” and “KPI”are used interchangeably throughout this disclosure. Data lineagerepository 118 may include information relating to how each data table160 of the database 156 is related to other data tables 160 of thedatabase 156. For example, data lineage information relating to a firstdata table may indicate that a second data table depends upon the firstdata table to calculate values relating to at least one data field/datatype in the second data table. As described below, data lineageinformation relating to a data table 160 from the data lineagerepository 118 may be used to determine how critical the data table maybe for one or more applications in the target computing environment 170.Query execution history 120 relating to a data table 160 may includeinformation relating to queries processing in the data table 160. Copymanager 110 may be configured to derive relations between data tables160 based on query execution histories 120 relating to the data tables160. For example, query execution history 120 may indicate that datafrom several data tables 160 was accessed to process a query, indicatingthat those data tables 160 are related. In one embodiment, copy manager110 may be configured to derive relations between data tables 160 basedon query execution histories 120 when data lineage repository 118 isunavailable or otherwise does not include data lineage informationrelating to one or more data tables 160. In addition, table metadata 112may also include data logs relating to data tables 160 which can also beused to derive data lineage information relating to data tables 160.

User-defined metrics may include, but are not limited to, read weightage124, write weightage 126, data date range 128, KPI weightage 130 andstale data movement flag 132. Read weightage 124 assigned to a datatable 160 may include a numerical weight assigned to read operationsperformed in the data table 160. Write weightage 126 assigned to a datatable 160 may include a numerical weight assigned to write operationsperformed in the data table 160. KPI weightage 130 assigned to a datatable 160 may include a numerical weight assigned to the importanceindex 116 (e.g., KPI) of the data table 160. As further described below,each of the read weightage 124, write weightage 126 and KPI weightage130 decides how much influence the respective metric (e.g., readoperations, write operations and KPI respectively) has in deciding thevalue of a data table 160 in the target computing environment 170. Forexample, a user 192 of the target computing environment 170 may assign ahigher read weightage 124 to read intensive data tables 160 and mayassign a higher write weightage 126 to write intensive data tables 160.A user may assign the KPI weightage 130 to adjust the default KPI indexassigned to a data table 160. Data date range 128 defined for a datatable 160 may include a date range (and/or time range), wherein datafrom the data table 160 associated with the data range (and/or timerange) is to be copied to the target memory devices 174. For example, adata date range 128 defined for a data table 160 containing employeerecords for an organization, may specify that employee records relatingto employee joining dates within the past three months are to be copiedto the target memory devices 174. Defining a data date range 128 may beuseful especially for large data tables 160 when the target computingenvironment does not need all data from the data tables 160. Forexample, when the target computing environment 170 is a lower levelenvironment such as a test environment, all data from a large data tablemay not be needed to test certain features. In this case, a user 192 ofthe target computing environment 170 may define a data date range forthe data table 160 so that data relating to the data date range can becopied instead of copying the entire data table 160. Stale data movementflag 132 relating to a data table 160 may specify whether data from adata table 160 that has remained unchanged since a previous data copy tothe target memory devices 174 is to be copied again in a subsequent datacopy to the target memory devices 174. As described above, the targetcomputing environment 170 may undergo periodic data restorations fromthe source computing environment 150. Typically, a portion of the datastored in the database 156 may have remained unchanged between two datacopies. Generally, there is no need to re-copy data from a data table160 that has remained unchanged since a previous data copy. However, insome cases, the user 192 of the target computing environment 170 maywant unchanged data from the data table 160 to be copied again in asubsequent data copy, for example, to record unchanged status of thedata over several data copies. In such a case, a user 192 may set thestale data movement flag 132 for the data table 160 to indicate thatdata from the data table 160 that has remined unchanged since a previousdata copy is to be copied again to the target memory devices 174 in asubsequent data copy.

In one embodiment, when one or more of the user-defined metrics 122 arenot defined for a data table 160, pre-selected default values are setfor the respective user defined metrics 122. For example, when a readweightage 124, write weightage 126 or KPI weightage 130 is not definedby a user 192 for a particular data table 160, default weightages areassigned to these metrics.

Environment metadata 134 may include, but is not limited to, metadatarelating to the source computing environment 150 as well as the targetcomputing environment 170. Metadata relating to the source computingenvironment 150 may include a size of each data table 160. Metadatarelating to the target computing environment 170 may include a number oftarget memory devices 174, size of each target memory device 174, memoryspace available in the target memory devices 174 to store data andamount of space needed in each target memory device 174 for operationalpurposes. Copy manager 110 may be configured to determine based on theenvironment metadata 134 that the target memory devices 174 haveinsufficient memory space to receive and store the entire database 156(or otherwise all data) stored in the source memory devices 154.

Copy manager 110 may be configured to determine a plurality of tablevaluation metrics 136 for each data table 160 based on one or more ofthe table metadata 112 and the user-defined metrics 122. For example,copy manager 110 may be configured to calculate a criticality index 144for each data table 160 based on data lineage information relating tothe data table 160. The criticality index 144 of a data table 160 isindicative of how critical the data table 160 is to one or more softwareapplications in the target computing environment 170, based on whetherand how many other data tables 160 depend on the data table 160. Copymanager 110 may obtain data lineage information relating to each datatable 160 from data lineage repository 118 or may determine the datalineage information from query execution history 120 of the data table160. For example, for each data table 160, copy manager 110 may beconfigured to determine whether one or more other data tables 160 dependon the data in the data table 160 for calculating a value of at leastone data field. Copy manager 110 may be configured to calculate thecriticality index 144 of the data table 160 based on a KPI index 116 ofthe data table 160 and the criticality index 144 of the one or moreother data tables 160 that depend on the data table 160, wherein thecriticality index 144 of a data table 160 is higher when one or moreother data tables 160 depend on the data table 160 as compared to thecriticality index of the data table 160 when no other data tables dependon the data table 160.

FIG. 2 illustrates an example calculation of criticality index 144 fordata tables 160 based on data lineage information of data tables 160, inaccordance with one or more embodiments of the present disclosure. FIG.2 shows four data tables named Table 1, Table 2, Table 3 and Table 4.Table 1 includes three columns/fields named as ColA, ColB and ColC.Table 2 includes four columns/fields named as ColW, ColX, ColY and ColZ.Table 3 includes two columns/fields named as ColG and ColH. Table 4includes two columns/fields named as ColS and ColT. As shown, ColZ ofTable 2 is calculated as an addition of ColA and ColC of Table 1. Thismay mean that a value of any data field in ColZ of Table 2 is calculatedas a summation of values in respective data fields in ColA and ColC ofTable 1. Similarly, ColH of Table 3 is calculated as a sum of ColZ andColX. The pre-assigned KPIs (e.g., same as importance index 116) ofTables 1, 2, 3 and 4 are 3, 4, 3 and 4 respectively. As shown, KPI percolumn (shown as KPI/Column) for each data table may be calculated asthe KPI of the data table divided by the number of columns in the table.The criticality index for each of the data tables may be calculated as:

((KPI/column)*number of columns)+Criticality Index of each dependentdata table

For example, the criticality index of Table 3 may be calculated as(((3/2)*2)/10)=0.3. As no other data table depends on Table 3, thecriticality index of Table 3 is based only on its own KPI. Criticalityindex of Table 2 may be calculated as (((4/4)*4)/10)+(Criticality Indexof Table 3)=0.7. The criticality index of Table 3 is added here as Table3 depends on Table 2. Similarly, criticality index of Table 1 may becalculated as ((3/3)*3)/10+(Criticality Index of Table 2)=1, as Table 2depends on Table 1.

Copy manager 110 may be configured to calculate Read per MB 138 for eachdata table 160 as No. of Read Operations performed in the data tabledivided by the table size 115 of the data table 160. Copy manager 110may be configured to calculate Writes per MB 140 for each data table 160as No. of Write Operations performed in the data table divided by thetable size 115 of the data table 160. Copy manager 110 may be configuredto obtain the No. of Read Operations and No. of Write Operationsperformed in the data table 160 from the interaction statistics 114 ofthe data table 160. As shown in FIG. 1 , table valuation metrics 136 foreach data table 160 may also include the KPI 116 of the data table 160.As further described below, criticality index 144 of a data table 160may be used in calculating a value coefficient 147 of the data table 160and to eventually assign a priority index 149 to the data table 160,wherein a higher criticality index 144 results in a higher valuecoefficient 147 and eventually a higher priority index 149 for the datatable 160. A higher value coefficient 147 and priority index 149 of adata table 160 increases the likelihood that the data files 158 and filegroups 159 containing data relating to the data table 160 are copied tothe target memory devices 174. Thus, the criticality index 144 helpspreserve data lineage in the target computing environment 170 byinfluencing prioritization of data tables 160 upon which one or moreother data tables 160 depend from.

Copy manager 110 may be configured to calculate a value coefficient 147based on one or more of the table valuation metrics 136.

FIGS. 3A and 3B illustrate an example calculation of value coefficients147 for data tables 160, in accordance with one or more embodiments ofthe present disclosure.

FIG. 3A shows five data tables 160 named as Table A, B, C D and E. FIG.3A also shows the table valuation metrics 136 for each data table 160including read per MB 138, write per MB 140, KPI index 116 and thecalculated criticality index 144 for the data table 160. FIG. 3B showsuser-defined metrics 122 that applies to all the five tables A-E. Asshown, Read weightage 124 is 0.4, Write weightage 126 is 0.5 and KPIweightage 130 is 0.1. Further, as shown, the stale data movement flag132 is set to “Y”, meaning that unchanged data from a previous data copyis to be copied again. It may be noted that while FIG. 3B shows commonuser-defined metrics 122 assigned to all data tables, it may beappreciated that one or more of the data tables may have customuser-defined metrics 122 that are different from the user-definedmetrics 122 of one or more other data tables.

A value coefficient 147 may be calculated for each of the data tables160 as,

Value Coefficient=[(Read Per MB*Read Weightage)+(Write Per MB*WriteWeightage)+(KPI Index*KPI Weightage)]/Criticality Index

FIG. 3A shows the calculated value coefficients 147 for each of theTables A-E based on the above equation. As shown value coefficients 147for Tables A-E are 0.544275347, 0.440070869, 0.643952764 and 0.48respectively.

Once the value coefficients 147 have been calculated for the data tables160, copy manager 110 may be configured to assign a priority index 149to each data table 160 based on the value coefficient 147 of the datatable 160, wherein a higher priority index 149 is assigned to a datatable 160 having a higher value coefficient 147. As shown in FIG. 3A,priority indices of 1-5 have been assigned to the data tables A-E, withthe highest priority index of “1” assigned to Table D having the highestvalue coefficient 147 and the lowest priority index of “5” assigned toTable C having the lowest value coefficient 147.

Once a value coefficient 147 and priority index 149 has been determinedfor each data table 160, copy manager 110 may be configured to schedulecopying of the file groups 159 and data files 158 in order of thepriority indices 149 assigned to respective data tables 160 startingwith data files 158 containing data relating to data tables 160 with thehighest assigned priority indices 149. In one embodiment, copy manager110 may be configured to re-arrange the file groups 159 such that datafiles 158 containing data relating to data tables 160 with higherpriority indices 149 are assigned to file groups 159 scheduled to becopied earlier to the target memory devices 174, and data files 158containing data relating to data tables 160 with lower priority indices149 are assigned to file groups 159 scheduled to be copied later to thetarget memory devices 174. For example, file groups FG1-FGn may beplaced in a copy queue in numerical order with FG1 scheduled to becopied first and the FGn scheduled to be copied last. Copy manager 110may be configured to assign data files 158 to the file groups FG1-FGnbased on the priority indices 149 of data tables 160 for which the datafiles 158 hold data, wherein data files 158 containing data relating todata tables 160 having the highest priority indices 149 are assigned toFG1 and data files 158 containing data relating to data tables 160having the lowest priority indices 149 are assigned to FGn. Re-arrangingthe file groups in the copy queue based on the priority indices 149 ofthe data tables 160 may ensure that the most critical data tables arecopied first to the target memory devices 174. Thus, for example, evenwhen all file groups FG1-FGn cannot be copied to the target memorydevices 174 as a result of insufficient storage space in the targetmemory devices 174, there is a high likelihood that most or all criticaldata files 158 and corresponding data tables 160 are copied to beforethe target memory devices 174 runs out of memory. As shown in FIG. 1 ,target memory devices 174 have received and stored a copy of thedatabase 156 (shown as DB1_copy) with file groups FG1-FGn-x having beencopied when the target memory devices 174 run out of memory. In thiscase, n-x refer to the number of file groups 159 and/or data files 158that were not copied as a result of insufficient memory space in thetarget memory devices 174.

In one or more embodiments, when a data date range 128 has been definedfor a data table 160, data files 158 that contain data relating to thedefined data date range 128 for the data table 160 are prioritized overdata files 158 that contain data that is outside the defined data daterange 128 for the data table 160. For example, only data files 158 thatcontain data relating to the defined data date range 128 for the datatable 160 are assigned to the file groups 159 based on the priorityindex 149 of the data table 160, so that data from the data table 160that is outside the defined data date range 128 is not copied to thetarget memory devices 174. In additional or alternative embodiments,when the stale data movement flag 132 for a data table 160 is set toindicate that unchanged data from a the data table 160 from a previouscopy is not to be copied again, copy manager 110 may be configured toskip assigning data files 158 to file groups 159 that contain unchangeddata from the data table 160, so that unchanged data is not copied againto the target memory devices 174. These measures may further help ensurethat most critical data files 158 are copied to the target memorydevices 174.

In one or more alternative or additional embodiments, copy manager 110may use information relating to a previous data restoration or data copyto re-arrange the file groups 159. For example, copy manager 110 mayhave access to re-organization archive 148 that includes informationrelating to re-organization of file groups 159 from a previous data copybetween the source computing environment 150 and the target computingenvironment 170. For example, re-organization archive 148 may includeinformation such as data table name/data file name, database name,original file group from which the data file/data table was moved,target file group to which the data file/data table was moved for thedata copy, target drive/disk to which the file group was moved etc. Copymanager 110 may be configured to re-arrange the file groups 159 at leastpartially based on information from the re-organization archive 148relating to how one or more data files/data tables were re-assignedduring a previous data copy. For example, copy manager 110 may beconfigured to assign a data file 158 to a file group 159 based oninformation relating to how the data file 158 was re-assigned in aprevious data copy. In one embodiment, copy manager 110 may assign adata file/data table to the same file group 159 it was assigned in aprevious data copy.

In one or more alternative or additional embodiments, copy manager 110may be configured to generate an output file 146 that includesinformation relating to how the file groups 159 are to be re-arrangedfor a data copy between the source computing environment 150 and thetarget computing environment 170. The output file 146 may include foreach data file/data table, table name/data file name, table size, sourcedrive in the source computing environment 150, target drive in thetarget computing environment 170, source file group 159 the datafile/data table was assigned and a target file group 159 the datafile/data table is to be assigned to for the data copy. The data copymay be performed (e.g., automatically by the copy manager 110 ormanually by a user 192) based on the information in the output file 146.

FIG. 4 is a flowchart of an example method 400 for managing a data copyfrom a source computing environment 150 to a target computingenvironment 170, in accordance with one or more embodiments of thepresent disclosure. Method 400 may be performed by the copy manager 110as shown in FIG. 1 and described above.

At operation 402, copy manager 110 receives a command or request to copya plurality of data files from one or more source memory devices 154 toone or more target memory devices 174.

As described above, the target computing environment 170 may undergoperiodic restoration which includes copying data stored in the sourcememory devices 154 to the target memory devices 174. Each restorationevent essentially creates an image of the source computing environment150 in the target computing environment 170. For example, the sourcecomputing environment 150 may be a production environment of anorganization and the target computing environment may be a lower levelenvironment such as a development environment or test environment.However, in some cases, the target computing environment 170 may haveinsufficient storage capacity within the target memory devices 174 toaccommodate the entire data stored in the source memory devices 154during a data copy. Copy manager 110 may be configured to manage a datacopy or data restoration from the source computing environment 150 tothe target computing environment 170. Copy manager 110 may receive arequest for a data restore or data copy from the source computingenvironment 150 to the target computing environment 170. The request forthe data copy may be generated by a user 192 using a user device 190.The user 192 generating the request may be an administrator of thesource computing environment 150 and/or the target computing environment170 or any other user 192 with appropriate credentials to make such arequest.

At operation 404, in response to receiving the request for data copy,copy manager 110 may determine whether the target memory devices 174have sufficient memory space to receive and store the entire database156 (or otherwise all data) stored in the source memory devices 154.Copy manager 110 may have access to environment metadata 134 includingmetadata relating to the source computing environment 150 and the targetcomputing environment 170. The environment metadata 134 may includeinformation relating to a size of each of the source memory devices 154,a size of each of the target memory devices 174, a size of each datatable 160 of the database 156, and memory space needed in each of thetarget memory devices 174 for operational purposes. Copy manager 110 maybe configured to calculate a total amount of data to be copied from thesource memory devices 154 to the target memory devices 174 based on theindividual data table sizes of the data tables 160. For example, copymanager 110 may add the sizes of all data tables 160 (e.g., obtainedfrom the environment metadata 134) to determine the total data size tobe copied to the target memory devices 174. Copy manager 110 may beconfigured to determine a combined memory space available at the targetmemory devices 174 (e.g., when the target memory devices includemultiple storage disks) to store data received from the source memorydevices 154. For example, copy manager 110 may calculate the availablememory at the target memory devices 174 by subtracting the total memoryspace needed at the target memory devices 174 for operational purposesfrom the total combined memory space of target memory devices 174. Copymanager 110 may be configured to compare the total data size to becopied from the source memory devices 154 with the total memory spaceavailable at the target memory devices 174 to store the data. When thememory space available at the target memory devices 174 is less than thetotal data size to be copied from the source memory devices 154, copymanager may be configured to determine that the target memory devices174 have insufficient memory space to receive and store the entiredatabase 156 (or otherwise all data) stored in the source memory devices154.

When copy manager 110 determines that the target memory devices 174 havesufficient memory space to receive and store the entire database 156 (orotherwise all data) stored in the source memory devices 154, method 400proceeds to operation 406 where copy manager 110 copies all data files158 and file groups 159 from the source memory devices 154 to the targetmemory devices 174. However, when copy manager 110 determines that thetarget memory devices 174 have insufficient memory space to receive andstore the entire database 156 (e.g. the plurality of data files 158 andfile groups 159) stored in the source memory devices 154, method 400proceeds to operation 408,

At operation 408, copy manager 110 calculates a value coefficient foreach data table 160 of the database 156.

In response to determining that target memory devices 174 haveinsufficient memory space to store the entire database 156 (or otherwiseall data) stored in the source memory devices 154, copy manager 110 maybe configured to assign priorities (e.g., priority index 149 as shown inFIG. 1 ) to data tables 160 based on pre-defined metrics, and copy datafiles 158 and file groups 159 relating to the data tables 160 in theorder of respective priorities assigned to the data tables 160 startingwith data files 158 having data for data tables 160 with the highestassigned priorities. Copy manager 110 may assign priorities to each datatable 160 of the database 156 based on a relative importance of the datatable 160 (as determined based on the pre-defined metrics) among aplurality of data tables 160 defined for the database 156. Thistechnique may ensure that data relating to the most critical data tables160 is copied prior to data relating relatively non-critical data tables160. As further described below, one or more of the pre-defined metricsmay be indicative of or can be used to determine the importance of eachdata table 160 in the context of the target computing environment 170.Each pre-defined metric associated with a data table 160 may have adefault value based on the nature of data contained in the data table160 or a user-defined value to suit particular needs of the targetcomputing environment 170.

Copy manager 110 may have access to a plurality of pre-defined metricsrelating to each data table 160. As shown in FIG. 1 , the pre-definedmetrics may include, but are not limited to, table metadata 112,user-defined metrics 122 and environment metadata 134. Table metadata112 may include, but is limited to, table interaction statistics 114,table size 115, importance index 116, data lineage repository 118 andquery execution history 120. Table interaction statistics 114 relatingto a data table 160 may include information regarding an amount of oneor more types of interactions performed in relation to the data table160. For example, table interaction statistics 114 may include a totalnumber of read operations performed in the data table 160 and a totalnumber of write operations performed in the data table 160. Table size115 may include a size of a data table 160 indicative of a total memoryspace occupied by the data table 160 in the source memory devices 154.Importance index 116 relating to a data table 160 may be indicative ofan importance of the data table 160 in the source computing environment150. Importance index 116 may include a Key Performance Indicator (KPI)index typically assigned to the data table 160 based on how importantthe data table is for the source computing environment 150 and/or thetarget computing environment 170. The terms “importance index” and “KPI”are used interchangeably throughout this disclosure. Data lineagerepository 118 may include information relating to how each data table160 of the database 156 is related to other data tables 160 of thedatabase 156. For example, data lineage information relating to a firstdata table may indicate that a second data table depends upon the firstdata table to calculate values relating to at least one data field/datatype in the second data table. As described below, data lineageinformation relating to a data table 160 from the data lineagerepository 118 may be used to determine how critical the data table maybe for one or more applications in the target computing environment 170.Query execution history 120 relating to a data table 160 may includeinformation relating to queries processing in the data table 160. Copymanager 110 may be configured to derive relations between data tables160 based on query execution histories 120 relating to the data tables160. For example, query execution history 120 may indicate that datafrom several data tables 160 was accessed to process a query, indicatingthat those data tables 160 are related. In one embodiment, copy manager110 may be configured to derive relations between data tables 160 basedon query execution histories 120 when data lineage repository 118 isunavailable or otherwise does not include data lineage informationrelating to one or more data tables 160. In addition, table metadata 112may also include data logs relating to data tables 160 which can also beused to derive data lineage information relating to data tables 160.

User-defined metrics may include, but are not limited to, read weightage124, write weightage 126, data date range 128, KPI weightage 130 andstale data movement flag 132. Read weightage 124 assigned to a datatable 160 may include a numerical weight assigned to read operationsperformed in the data table 160. Write weightage 126 assigned to a datatable 160 may include a numerical weight assigned to write operationsperformed in the data table 160. KPI weightage 130 assigned to a datatable 160 may include a numerical weight assigned to the importanceindex 116 (e.g., KPI) of the data table 160. As further described below,each of the read weightage 124, write weightage 126 and KPI weightage130 decides how much influence the respective metric (e.g., readoperations, write operations and KPI respectively) has in deciding thevalue of a data table 160 in the target computing environment 170. Forexample, a user 192 of the target computing environment 170 may assign ahigher read weightage 124 to read intensive data tables 160 and mayassign a higher write weightage 126 to write intensive data tables 160.A user may assign the KPI weightage 130 to adjust the default KPI indexassigned to a data table 160. Data date range 128 defined for a datatable 160 may include a date range (and/or time range), wherein datafrom the data table 160 associated with the data range (and/or timerange) is to be copied to the target memory devices 174. For example, adata date range 128 defined for a data table 160 containing employeerecords for an organization, may specify that employee records relatingto employee joining dates within the past three months are to be copiedto the target memory devices 174. Defining a data date range 128 may beuseful especially for large data tables 160 when the target computingenvironment does not need all data from the data tables 160. Forexample, when the target computing environment 170 is a lower levelenvironment such as a test environment, all data from a large data tablemay not be needed to test certain features. In this case, a user 192 ofthe target computing environment 170 may define a data date range forthe data table 160 so that data relating to the data date range can becopied instead of copying the entire data table 160. Stale data movementflag 132 relating to a data table 160 may specify whether data from adata table 160 that has remained unchanged since a previous data copy tothe target memory devices 174 is to be copied again in a subsequent datacopy to the target memory devices 174. As described above, the targetcomputing environment 170 may undergo periodic data restorations fromthe source computing environment 150. Typically, a portion of the datastored in the database 156 may have remained unchanged between two datacopies. Generally, there is no need to re-copy data from a data table160 that has remained unchanged since a previous data copy. However, insome cases, the user 192 of the target computing environment 170 maywant unchanged data from the data table 160 to be copied again in asubsequent data copy, for example, to record unchanged status of thedata over several data copies. In such a case, a user 192 may set thestale data movement flag 132 for the data table 160 to indicate thatdata from the data table 160 that has remined unchanged since a previousdata copy is to be copied again to the target memory devices 174 in asubsequent data copy.

In one embodiment, when one or more of the user-defined metrics 122 arenot defined for a data table 160, pre-selected default values are setfor the respective user defined metrics 122. For example, when a readweightage 124, write weightage 126 or KPI weightage 130 is not definedby a user 192 for a particular data table 160, default weightages areassigned to these metrics.

Environment metadata 134 may include, but is not limited to, metadatarelating to the source computing environment 150 as well as the targetcomputing environment 170. Metadata relating to the source computingenvironment 150 may include a size of each data table 160. Metadatarelating to the target computing environment 170 may include a number oftarget memory devices 174, size of each target memory device 174, memoryspace available in the target memory devices 174 to store data andamount of space needed in each target memory device 174 for operationalpurposes. Copy manager 110 may be configured to determine based on theenvironment metadata 134 that the target memory devices 174 haveinsufficient memory space to receive and store the entire database 156(or otherwise all data) stored in the source memory devices 154.

Copy manager 110 may be configured to determine a plurality of tablevaluation metrics 136 for each data table 160 based on one or more ofthe table metadata 112 and the user-defined metrics 122. For example,copy manager 110 may be configured to calculate a criticality index 144for each data table 160 based on data lineage information relating tothe data table 160. The criticality index 144 of a data table 160 isindicative of how critical the data table 160 is to one or more softwareapplications in the target computing environment 170, based on whetherand how many other data tables 160 depend on the data table 160. Copymanager 110 may obtain data lineage information relating to each datatable 160 from data lineage repository 118 or may determine the datalineage information from query execution history 120 of the data table160. For example, for each data table 160, copy manager 110 may beconfigured to determine whether one or more other data tables 160 dependon the data in the data table 160 for calculating a value of at leastone data field. Copy manager 110 may be configured to calculate thecriticality index 144 of the data table 160 based on a KPI index 116 ofthe data table 160 and the criticality index 144 of the one or moreother data tables 160 that depend on the data table 160, wherein thecriticality index 144 of a data table 160 is higher when one or moreother data tables 160 depend on the data table 160 as compared to thecriticality index of the data table 160 when no other data tables dependon the data table 160. FIG. 2 illustrates an example calculation ofcriticality index 144 for data tables 160 based on data lineageinformation of data tables 160 as described above.

Copy manager 110 may be configured to calculate Read per MB 138 for eachdata table 160 as No. of Read Operations performed in the data tabledivided by the table size 115 of the data table 160. Copy manager 110may be configured to calculate Writes per MB 140 for each data table 160as No. of Write Operations performed in the data table divided by thetable size 115 of the data table 160. Copy manager 110 may be configuredto obtain the No. of Read Operations and No. of Write Operationsperformed in the data table 160 from the interaction statistics 114 ofthe data table 160. As shown in FIG. 1 , table valuation metrics 136 foreach data table 160 may also include the KPI 116 of the data table 160.As further described below, criticality index 144 of a data table 160may be used in calculating a value coefficient 147 of the data table 160and to eventually assign a priority index 149 to the data table 160,wherein a higher criticality index 144 results in a higher valuecoefficient 147 and eventually a higher priority index 149 for the datatable 160. A higher value coefficient 147 and priority index 149 of adata table 160 increases the likelihood that the data files 158 and filegroups 159 containing data relating to the data table 160 are copied tothe target memory devices 174. Thus, the criticality index 144 helpspreserve data lineage in the target computing environment 170 byinfluencing prioritization of data tables 160 upon which one or moreother data tables 160 depend from.

Copy manager 110 may be configured to calculate a value coefficient 147based on one or more of the table valuation metrics 136. FIGS. 3A and 3Billustrate an example calculation of value coefficients 147 for datatables 160 as described above.

At operation 410, copy manager 110 assigns a priority index 149 to eachdata table 160 based on the calculated value coefficient 147 of the datatable 160, wherein a higher priority index 149 is assigned to a datatable 160 having a higher value coefficient 147.

As described above, once the value coefficients 147 have been calculatedfor each of the data tables 160, copy manager 110 may be configured toassign a priority index 149 to each data table 160 based on the valuecoefficient 147 of the data table 160, wherein a higher priority index149 is assigned to a data table 160 having a higher value coefficient147. As shown in FIG. 3A, priority indices of 1-5 have been assigned tothe data tables A-E, with the highest priority index of “1” assigned toTable D having the highest value coefficient 147 and the lowest priorityindex of “5” assigned to Table C having the lowest value coefficient147.

At operation 412, copy manager 110 re-arranges the file groups 159 byassigning the data files 158 to the file groups 159 based on thepriority index 149 of the data tables 160 associated with the data files158, wherein a data file 158 that includes data associated with a datatable 160 with a higher priority index 149 is assigned to one or morefile groups 159 that are earlier in a copy queue for copying to thetarget memory devices 174.

As described above, once a value coefficient 147 and priority index 149has been determined for each data table 160, copy manager 110 may beconfigured to schedule copying of the file groups 159 and data files 158in order of the priority indices 149 assigned to respective data tables160 starting with data files 158 containing data relating to data tables160 with the highest assigned priority indices 149. In one embodiment,copy manager 110 may be configured to re-arrange the file groups 159such that data files 158 containing data relating to data tables 160with higher priority indices 149 are assigned to file groups 159scheduled to be copied earlier to the target memory devices 174, anddata files 158 containing data relating to data tables 160 with lowerpriority indices 149 are assigned to file groups 159 scheduled to becopied later to the target memory devices 174. For example, file groupsFG1-FGn may be placed in a copy queue in numerical order with FG1scheduled to be copied first and the FGn scheduled to be copied last.Copy manager 110 may be configured to assign data files 158 to the filegroups FG1-FGn based on the priority indices 149 of data tables 160 forwhich the data files 158 hold data, wherein data files 158 containingdata relating to data tables 160 having the highest priority indices 149are assigned to FG1 and data files 158 containing data relating to datatables 160 having the lowest priority indices 149 are assigned to FGn.Re-arranging the file groups in the copy queue based on the priorityindices 149 of the data tables 160 may ensure that the most criticaldata tables are copied first to the target memory devices 174. Thus, forexample, even when all file groups FG1-FGn cannot be copied to thetarget memory devices 174 as a result of insufficient storage space inthe target memory devices 174, there is a high likelihood that most orall critical data files 158 and corresponding data tables 160 are copiedto before the target memory devices 174 runs out of memory. As shown inFIG. 1 , target memory devices 174 have received and stored a copy ofthe database 156 (shown as DB1_copy) with file groups FG1-FGn-x havingbeen copied when the target memory devices 174 run out of memory. Inthis case, n-x refer to the number of file groups 159 and/or data files158 that were not copied as a result of insufficient memory space in thetarget memory devices 174.

In one or more embodiments, when a data date range 128 has been definedfor a data table 160, data files 158 that contain data relating to thedefined data date range 128 for the data table 160 are prioritized overdata files 158 that contain data that is outside the defined data daterange 128 for the data table 160. For example, only data files 158 thatcontain data relating to the defined data date range 128 for the datatable 160 are assigned to the file groups 159 based on the priorityindex 149 of the data table 160, so that data from the data table 160that is outside the defined data date range 128 is not copied to thetarget memory devices 174. In additional or alternative embodiments,when the stale data movement flag 132 for a data table 160 is set toindicate that unchanged data from a the data table 160 from a previouscopy is not to be copied again, copy manager 110 may be configured toskip assigning data files 158 to file groups 159 that contain unchangeddata from the data table 160, so that unchanged data is not copied againto the target memory devices 174. These measures may further help ensurethat most critical data files 158 are copied to the target memorydevices 174.

At operation 414, copy manager 110 schedules copying of the re-arrangeddata file groups 159 to the target memory devices 174 according to thecopy queue.

FIG. 5 is a flowchart of an example method 500 for valuating a datatable 160, in accordance with one or more embodiments of the presentdisclosure. Method 500 may be performed by the copy manager 110 as shownin FIG. 1 and described above.

At operation 502, copy manager 110 receives a command or request to copya plurality of data files from one or more source memory devices 154 toone or more target memory devices 174.

As described above, the target computing environment 170 may undergoperiodic restoration which includes copying data stored in the sourcememory devices 154 to the target memory devices 174. Each restorationevent essentially creates an image of the source computing environment150 in the target computing environment 170. For example, the sourcecomputing environment 150 may be a production environment of anorganization and the target computing environment may be a lower levelenvironment such as a development environment or test environment.However, in some cases, the target computing environment 170 may haveinsufficient storage capacity within the target memory devices 174 toaccommodate the entire data stored in the source memory devices 154during a data copy. Copy manager 110 may be configured to manage a datacopy or data restoration from the source computing environment 150 tothe target computing environment 170. Copy manager 110 may receive arequest for a data restore or data copy from the source computingenvironment 150 to the target computing environment 170. The request forthe data copy may be generated by a user 192 using a user device 190.The user 192 generating the request may be an administrator of thesource computing environment 150 and/or the target computing environment170 or any other user 192 with appropriate credentials to make such arequest.

At operation 504, in response to receiving the request for data copy,copy manager 110 may determine whether the target memory devices 174have sufficient memory space to receive and store the entire database156 (or otherwise all data) stored in the source memory devices 154.Copy manager 110 may have access to environment metadata 134 includingmetadata relating to the source computing environment 150 and the targetcomputing environment 170. The environment metadata 134 may includeinformation relating to a size of each of the source memory devices 154,a size of each of the target memory devices 174, a size of each datatable 160 of the database 156, and memory space needed in each of thetarget memory devices 174 for operational purposes. Copy manager 110 maybe configured to calculate a total amount of data to be copied from thesource memory devices 154 to the target memory devices 174 based on theindividual data table sizes of the data tables 160. For example, copymanager 110 may add the sizes of all data tables 160 (e.g., obtainedfrom the environment metadata 134) to determine the total data size tobe copied to the target memory devices 174. Copy manager 110 may beconfigured to determine a combined memory space available at the targetmemory devices 174 (e.g., when the target memory devices includemultiple storage disks) to store data received from the source memorydevices 154. For example, copy manager 110 may calculate the availablememory at the target memory devices 174 by subtracting the total memoryspace needed at the target memory devices 174 for operational purposesfrom the total combined memory space of target memory devices 174. Copymanager 110 may be configured to compare the total data size to becopied from the source memory devices 154 with the total memory spaceavailable at the target memory devices 174 to store the data. When thememory space available at the target memory devices 174 is less than thetotal data size to be copied from the source memory devices 154, copymanager may be configured to determine that the target memory devices174 have insufficient memory space to receive and store the entiredatabase 156 (or otherwise all data) stored in the source memory devices154.

When copy manager 110 determines that the target memory devices 174 havesufficient memory space to receive and store the entire database 156 (orotherwise all data) stored in the source memory devices 154, method 500proceeds to operation 506 where copy manager 110 copies all data files158 and file groups 159 from the source memory devices 154 to the targetmemory devices 174. However, when copy manager 110 determines that thetarget memory devices 174 have insufficient memory space to receive andstore the entire database 156 (e.g. the plurality of data files 158 andfile groups 159) stored in the source memory devices 154, method 500proceeds to operation 508.

In response to determining that target memory devices 174 haveinsufficient memory space to store the entire database 156 (or otherwiseall data) stored in the source memory devices 154, copy manager 110 maybe configured to assign priorities (e.g., priority index 149 as shown inFIG. 1 ) to data tables 160 based on pre-defined metrics, and copy datafiles 158 and file groups 159 relating to the data tables 160 in theorder of respective priorities assigned to the data tables 160 startingwith data files 158 having data for data tables 160 with the highestassigned priorities. Copy manager 110 may assign priorities to each datatable 160 of the database 156 based on a relative importance of the datatable 160 (as determined based on the pre-defined metrics) among aplurality of data tables 160 defined for the database 156. Thistechnique may ensure that data relating to the most critical data tables160 is copied prior to data relating relatively non-critical data tables160. As further described below, one or more of the pre-defined metricsmay be indicative of or can be used to determine the importance of eachdata table 160 in the context of the target computing environment 170.Each pre-defined metric associated with a data table 160 may have adefault value based on the nature of data contained in the data table160 or a user-defined value to suit particular needs of the targetcomputing environment 170.

Copy manager 110 may have access to a plurality of pre-defined metricsrelating to each data table 160. As shown in FIG. 1 , the pre-definedmetrics may include, but are not limited to, table metadata 112,user-defined metrics 122 and environment metadata 134. Table metadata112 may include, but is limited to, table interaction statistics 114,table size 115, importance index 116, data lineage repository 118 andquery execution history 120. Table interaction statistics 114 relatingto a data table 160 may include information regarding an amount of oneor more types of interactions performed in relation to the data table160. For example, table interaction statistics 114 may include a totalnumber of read operations performed in the data table 160 and a totalnumber of write operations performed in the data table 160. Table size115 may include a size of a data table 160 indicative of a total memoryspace occupied by the data table 160 in the source memory devices 154.Importance index 116 relating to a data table 160 may be indicative ofan importance of the data table 160 in the source computing environment150. Importance index 116 may include a Key Performance Indicator (KPI)index typically assigned to the data table 160 based on how importantthe data table is for the source computing environment 150 and/or thetarget computing environment 170. The terms “importance index” and “KPI”are used interchangeably throughout this disclosure. Data lineagerepository 118 may include information relating to how each data table160 of the database 156 is related to other data tables 160 of thedatabase 156. For example, data lineage information relating to a firstdata table may indicate that a second data table depends upon the firstdata table to calculate values relating to at least one data field/datatype in the second data table. As described below, data lineageinformation relating to a data table 160 from the data lineagerepository 118 may be used to determine how critical the data table maybe for one or more applications in the target computing environment 170.Query execution history 120 relating to a data table 160 may includeinformation relating to queries processing in the data table 160. Copymanager 110 may be configured to derive relations between data tables160 based on query execution histories 120 relating to the data tables160. For example, query execution history 120 may indicate that datafrom several data tables 160 was accessed to process a query, indicatingthat those data tables 160 are related. In one embodiment, copy manager110 may be configured to derive relations between data tables 160 basedon query execution histories 120 when data lineage repository 118 isunavailable or otherwise does not include data lineage informationrelating to one or more data tables 160. In addition, table metadata 112may also include data logs relating to data tables 160 which can also beused to derive data lineage information relating to data tables 160.

User-defined metrics may include, but are not limited to, read weightage124, write weightage 126, data date range 128, KPI weightage 130 andstale data movement flag 132. Read weightage 124 assigned to a datatable 160 may include a numerical weight assigned to read operationsperformed in the data table 160. Write weightage 126 assigned to a datatable 160 may include a numerical weight assigned to write operationsperformed in the data table 160. KPI weightage 130 assigned to a datatable 160 may include a numerical weight assigned to the importanceindex 116 (e.g., KPI) of the data table 160. As further described below,each of the read weightage 124, write weightage 126 and KPI weightage130 decides how much influence the respective metric (e.g., readoperations, write operations and KPI respectively) has in deciding thevalue of a data table 160 in the target computing environment 170. Forexample, a user 192 of the target computing environment 170 may assign ahigher read weightage 124 to read intensive data tables 160 and mayassign a higher write weightage 126 to write intensive data tables 160.A user may assign the KPI weightage 130 to adjust the default KPI indexassigned to a data table 160. Data date range 128 defined for a datatable 160 may include a date range (and/or time range), wherein datafrom the data table 160 associated with the data range (and/or timerange) is to be copied to the target memory devices 174. For example, adata date range 128 defined for a data table 160 containing employeerecords for an organization, may specify that employee records relatingto employee joining dates within the past three months are to be copiedto the target memory devices 174. Defining a data date range 128 may beuseful especially for large data tables 160 when the target computingenvironment does not need all data from the data tables 160. Forexample, when the target computing environment 170 is a lower levelenvironment such as a test environment, all data from a large data tablemay not be needed to test certain features. In this case, a user 192 ofthe target computing environment 170 may define a data date range forthe data table 160 so that data relating to the data date range can becopied instead of copying the entire data table 160. Stale data movementflag 132 relating to a data table 160 may specify whether data from adata table 160 that has remained unchanged since a previous data copy tothe target memory devices 174 is to be copied again in a subsequent datacopy to the target memory devices 174. As described above, the targetcomputing environment 170 may undergo periodic data restorations fromthe source computing environment 150. Typically, a portion of the datastored in the database 156 may have remained unchanged between two datacopies. Generally, there is no need to re-copy data from a data table160 that has remained unchanged since a previous data copy. However, insome cases, the user 192 of the target computing environment 170 maywant unchanged data from the data table 160 to be copied again in asubsequent data copy, for example, to record unchanged status of thedata over several data copies. In such a case, a user 192 may set thestale data movement flag 132 for the data table 160 to indicate thatdata from the data table 160 that has remined unchanged since a previousdata copy is to be copied again to the target memory devices 174 in asubsequent data copy.

In one embodiment, when one or more of the user-defined metrics 122 arenot defined for a data table 160, pre-selected default values are setfor the respective user defined metrics 122. For example, when a readweightage 124, write weightage 126 or KPI weightage 130 is not definedby a user 192 for a particular data table 160, default weightages areassigned to these metrics.

Environment metadata 134 may include, but is not limited to, metadatarelating to the source computing environment 150 as well as the targetcomputing environment 170. Metadata relating to the source computingenvironment 150 may include a size of each data table 160. Metadatarelating to the target computing environment 170 may include a number oftarget memory devices 174, size of each target memory device 174, memoryspace available in the target memory devices 174 to store data andamount of space needed in each target memory device 174 for operationalpurposes. Copy manager 110 may be configured to determine based on theenvironment metadata 134 that the target memory devices 174 haveinsufficient memory space to receive and store the entire database 156(or otherwise all data) stored in the source memory devices 154.

Copy manager 110 may be configured to determine a plurality of tablevaluation metrics 136 for each data table 160 based on one or more ofthe table metadata 112 and the user-defined metrics 122.

At operation 508, copy manager calculates a number of read operationsper MB 138 in a data table 160 of a plurality of data tables 160 basedon a number of read operations performed in the data table and a tablesize 115 of the data table 160 The copy manager 110 further calculates anumber of write operations per MB 140 in the data table 160 based on anumber of write operations performed in the data table 160 and the tablesize 115 of the data table 160.

As described above, copy manager 110 may be configured to calculate Readper MB 138 for each data table 160 as No. of Read Operations performedin the data table divided by the table size 115 of the data table 160.Copy manager 110 may be configured to calculate Writes per MB 140 foreach data table 160 as No. of Write Operations performed in the datatable divided by the table size 115 of the data table 160. Copy manager110 may be configured to obtain the No. of Read Operations and No. ofWrite Operations performed in the data table 160 from the interactionstatistics 114 of the data table 160.

At operation 510, copy manager 110 obtains an importance index 116assigned to the data table 160, wherein the importance index 116 isindicative of an importance of the data table 160. The importance index116 may include a KPI assigned to the data table 160.

At operation 512, copy manager 110, weights the number of readoperations per MB, the number of write operations per MB and theimportance index 116 based on respective user-defined metrics. Asdescribed above, copy manager 110 may multiply the number of readoperations per MB 138 related to the data table 160 by the readweightage 124, multiply the number of write operations per MB 140related to the data table 160 by the write weightage 126, and multiplyKPI 116 with the KPI weightage 130.

At operation 514, copy manager 110 determines a criticality index 144for the data table 160 based on a relationship of the data table 160with other data tables 160 of the database 156.

As described above, copy manager 110 may be configured to calculate acriticality index 144 for each data table 160 based on data lineageinformation relating to the data table 160. The criticality index 144 ofa data table 160 is indicative of how critical the data table 160 is toone or more software applications in the target computing environment170, based on whether and how many other data tables 160 depend on thedata table 160. Copy manager 110 may obtain data lineage informationrelating to each data table 160 from data lineage repository 118 or maydetermine the data lineage information from query execution history 120of the data table 160. For example, for each data table 160, copymanager 110 may be configured to determine whether one or more otherdata tables 160 depend on the data in the data table 160 for calculatinga value of at least one data field. Copy manager 110 may be configuredto calculate the criticality index 144 of the data table 160 based on aKPI index 116 of the data table 160 and the criticality index 144 of theone or more other data tables 160 that depend on the data table 160,wherein the criticality index 144 of a data table 160 is higher when oneor more other data tables 160 depend on the data table 160 as comparedto the criticality index of the data table 160 when no other data tablesdepend on the data table 160. FIG. 2 illustrates an example calculationof criticality index 144 for data tables 160 based on data lineageinformation of data tables 160 as described above.

At operation 516, copy manager 110 calculates a value coefficient 147 ofthe data table 160 by adding the weighted read operations per MB, theweighted write operations per MB, the weighted importance index and thecriticality index 144 of the data table 160. Copy manager 110 may beconfigured to calculate a value coefficient 147 based on one or more ofthe table valuation metrics 136. FIGS. 3A and 3B illustrate an examplecalculation of value coefficients 147 for data tables 160 as describedabove.

At operation 518, copy manager 110 assigns a data file 158 that includesdata relating to the data table 160 to at least one of the file groups159 based on the value coefficient 147 of the data table 160, wherein adata file 158 that includes data associated with a data table 160 with ahigher value coefficient 147 is assigned to one or more file groups 159that are earlier in a copy queue for copying to the target memorydevices 174, wherein the file groups 159 are copied to the target memorydevices 174 according to the copy queue.

As described above, once the value coefficients 147 have been calculatedfor each of the data tables 160, copy manager 110 may be configured toassign a priority index 149 to each data table 160 based on the valuecoefficient 147 of the data table 160, wherein a higher priority index149 is assigned to a data table 160 having a higher value coefficient147. For example, as shown in FIG. 3A, priority indices of 1-5 have beenassigned to the data tables A-E, with the highest priority index of “1”assigned to Table D having the highest value coefficient 147 and thelowest priority index of “5” assigned to Table C having the lowest valuecoefficient 147.

Once a value coefficient 147 and priority index 149 has been determinedfor each data table 160, copy manager 110 may be configured to schedulecopying of the file groups 159 and data files 158 in order of thepriority indices 149 assigned to respective data tables 160 startingwith data files 158 containing data relating to data tables 160 with thehighest assigned priority indices 149. In one embodiment, copy manager110 may be configured to re-arrange the file groups 159 such that datafiles 158 containing data relating to data tables 160 with higherpriority indices 149 are assigned to file groups 159 scheduled to becopied earlier to the target memory devices 174, and data files 158containing data relating to data tables 160 with lower priority indices149 are assigned to file groups 159 scheduled to be copied later to thetarget memory devices 174. For example, file groups FG1-FGn may beplaced in a copy queue in numerical order with FG1 scheduled to becopied first and the FGn scheduled to be copied last. Copy manager 110may be configured to assign data files 158 to the file groups FG1-FGnbased on the priority indices 149 of data tables 160 for which the datafiles 158 hold data, wherein data files 158 containing data relating todata tables 160 having the highest priority indices 149 are assigned toFG1 and data files 158 containing data relating to data tables 160having the lowest priority indices 149 are assigned to FGn. Re-arrangingthe file groups in the copy queue based on the priority indices 149 ofthe data tables 160 may ensure that the most critical data tables arecopied first to the target memory devices 174. Thus, for example, evenwhen all file groups FG1-FGn cannot be copied to the target memorydevices 174 as a result of insufficient storage space in the targetmemory devices 174, there is a high likelihood that most or all criticaldata files 158 and corresponding data tables 160 are copied to beforethe target memory devices 174 runs out of memory. As shown in FIG. 1 ,target memory devices 174 have received and stored a copy of thedatabase 156 (shown as DB1_copy) with file groups FG1-FGn-x having beencopied when the target memory devices 174 run out of memory. In thiscase, n-x refer to the number of file groups 159 and/or data files 158that were not copied as a result of insufficient memory space in thetarget memory devices 174.

In one or more embodiments, when a data date range 128 has been definedfor a data table 160, data files 158 that contain data relating to thedefined data date range 128 for the data table 160 are prioritized overdata files 158 that contain data that is outside the defined data daterange 128 for the data table 160. For example, only data files 158 thatcontain data relating to the defined data date range 128 for the datatable 160 are assigned to the file groups 159 based on the priorityindex 149 of the data table 160, so that data from the data table 160that is outside the defined data date range 128 is not copied to thetarget memory devices 174. In additional or alternative embodiments,when the stale data movement flag 132 for a data table 160 is set toindicate that unchanged data from a the data table 160 from a previouscopy is not to be copied again, copy manager 110 may be configured toskip assigning data files 158 to file groups 159 that contain unchangeddata from the data table 160, so that unchanged data is not copied againto the target memory devices 174. These measures may further help ensurethat most critical data files 158 are copied to the target memorydevices 174.

FIG. 6 illustrates an example schematic diagram 600 of the copy manager110 illustrated in FIG. 1 , in accordance with one or more embodimentsof the present disclosure.

Copy manager 110 includes a processor 602, a memory 606, and a networkinterface 604. The copy manager 110 may be configured as shown in FIG. 6or in any other suitable configuration.

The processor 602 comprises one or more processors operably coupled tothe memory 606. The processor 602 is any electronic circuitry including,but not limited to, state machines, one or more central processing unit(CPU) chips, logic units, cores (e.g. a multi-core processor),field-programmable gate array (FPGAs), application specific integratedcircuits (ASICs), or digital signal processors (DSPs). The processor 602may be a programmable logic device, a microcontroller, a microprocessor,or any suitable combination of the preceding. The processor 602 iscommunicatively coupled to and in signal communication with the memory606. The one or more processors are configured to process data and maybe implemented in hardware or software. For example, the processor 602may be 8-bit, 16-bit, 32-bit, 64-bit or of any other suitablearchitecture. The processor 602 may include an arithmetic logic unit(ALU) for performing arithmetic and logic operations, processorregisters that supply operands to the ALU and store the results of ALUoperations, and a control unit that fetches instructions from memory andexecutes them by directing the coordinated operations of the ALU,registers and other components.

The one or more processors are configured to implement variousinstructions. For example, the one or more processors are configured toexecute instructions (e.g., copy manager instructions 608) to implementthe copy manager 110. In this way, processor 602 may be aspecial-purpose computer designed to implement the functions disclosedherein. In one or more embodiments, the copy manager 110 is implementedusing logic units, FPGAs, ASICs, DSPs, or any other suitable hardware.The copy manager 110 is configured to operate as described withreference to FIGS. 1-5 . For example, the processor 602 may beconfigured to perform at least a portion of the methods 400 and 500 asdescribed in FIGS. 4 and 5 respectively.

The memory 606 comprises one or more disks, tape drives, or solid-statedrives, and may be used as an over-flow data storage device, to storeprograms when such programs are selected for execution, and to storeinstructions and data that are read during program execution. The memory606 may be volatile or non-volatile and may comprise a read-only memory(ROM), random-access memory (RAM), ternary content-addressable memory(TCAM), dynamic random-access memory (DRAM), and static random-accessmemory (SRAM).

The memory 606 is operable to store the table metadata 112, user-definedmetrics 122, environment metadata 134, table valuation metrics 136,output file 146, re-organization archive 148, value coefficients 147,priority indices 149 and the copy manager instructions 608. The copymanager instructions 608 may include any suitable set of instructions,logic, rules, or code operable to execute the copy manager 110.

The network interface 604 is configured to enable wired and/or wirelesscommunications. The network interface 604 is configured to communicatedata between the copy manager 110 and other devices, systems, or domains(e.g. source computing environment 150, target computing environment170, user devices 190 etc.). For example, the network interface 604 maycomprise a Wi-Fi interface, a LAN interface, a WAN interface, a modem, aswitch, or a router. The processor 602 is configured to send and receivedata using the network interface 604. The network interface 604 may beconfigured to use any suitable type of communication protocol as wouldbe appreciated by one of ordinary skill in the art.

It may be noted that each of the source computing environment 150 (orcomponents 152 thereof), target computing environment 170 (or components172 thereof) and user devices 190 may be implemented similar to the copymanager 110. For example, each of the source computing environment 150(or components 152 thereof), target computing environment 170 (orcomponents 172 thereof) and user devices 190 may include a processor anda memory storing instructions to implement the respective functionalitywhen executed by the processor.

While several embodiments have been provided in the present disclosure,it should be understood that the disclosed systems and methods might beembodied in many other specific forms without departing from the spiritor scope of the present disclosure. The present examples are to beconsidered as illustrative and not restrictive, and the intention is notto be limited to the details given herein. For example, the variouselements or components may be combined or integrated in another systemor certain features may be omitted, or not implemented.

In addition, techniques, systems, subsystems, and methods described andillustrated in the various embodiments as discrete or separate may becombined or integrated with other systems, modules, techniques, ormethods without departing from the scope of the present disclosure.Other items shown or discussed as coupled or directly coupled orcommunicating with each other may be indirectly coupled or communicatingthrough some interface, device, or intermediate component whetherelectrically, mechanically, or otherwise. Other examples of changes,substitutions, and alterations are ascertainable by one skilled in theart and could be made without departing from the spirit and scopedisclosed herein.

To aid the Patent Office, and any readers of any patent issued on thisapplication in interpreting the claims appended hereto, applicants notethat they do not intend any of the appended claims to invoke 35 U.S.C. §112(f) as it exists on the date of filing hereof unless the words “meansfor” or “step for” are explicitly used in the particular claim.

1. A system comprising: one or more source memory devices of a sourcecomputing environment that store a database comprising a plurality ofdata files, wherein each of a plurality of data tables defined for thesource computing environment includes data from one or more of the datafiles; one or more target memory devices of a target computingenvironment that store at least a portion of the data files from the oneor more source memory devices; at least one processor communicativelycoupled to the one or more source memory devices and the one or moretarget memory devices and configured to: receive a command to copy theplurality of data files from the one or more source memory devices tothe one or more target memory devices; detect that the one or moretarget memory devices have insufficient memory space to accommodate theplurality of data files; calculate a value coefficient for each of theplurality of data tables, wherein the value coefficient corresponding toeach data table is indicative of a relative importance of the data tableamong the plurality of data tables, wherein the value coefficient isdetermined for each data table based at least on one or more interactionmetrics related to the data table indicating an amount of one or moretypes of data interactions performed in relation to the data table, animportance index assigned to the data table, user-defined numericalweights for each interaction metric and the importance index, and acriticality index based on a relationship of the data table with otherdata tables of the database; assign a priority index to each data tablebased on the calculated value coefficient of the data table, wherein ahigher priority is assigned to a data table having a higher valuecoefficient; order the data files in a copy queue for copying to thetarget memory devices based on the priority index of the data tablesassociated with the data files, wherein a data file that includes dataassociated with a data table with a higher priority index is orderedearlier in the copy queue; and schedule copy of the ordered data filesto the target memory devices according to the copy queue.
 2. The systemof claim 1, wherein: the interaction metrics related to each data tablecomprises one or more of a number of read operations per megabyte (MB)in the data table and a number of write operations per MB in the datatable; and the importance index assigned to each table comprises a KeyPerformance Indicator (KPI) index.
 3. The system of claim 2, wherein theat least one processor is configured to determine the criticality indexfor each data table by: determining that at least one other data tablecalculates a value in at least one data field based on data from thedata table; and calculating the criticality index of the data tablebased on the KPI index of the data table and the criticality index ofthe at least one other data table, wherein the criticality index of adata table is higher when one or more other data tables depend on thedata table as compared to the criticality index of the data table whenno other data tables depend on the data table.
 4. The system of claim 3,wherein the at least one processor is configured to determine the valuecoefficient of the data table by: weighting each of the interactionmetrics and the importance index based on the corresponding user definedweights; and determining the value coefficient of the data table byadding the weighted interaction metrics, the weighted importance indexand the criticality index of the data table.
 5. The system of claim 1,wherein the at least one processor is further configured to: obtain adata date range defined for the data in the data table; and copy one ormore data files having data associated with the data table that iswithin the data date range.
 6. The system of claim 1, wherein the atleast one processor is further configured to: obtain a stale datamovement flag for the data table, wherein the stale data movement flagindicates whether data from the data table that has remain unchangedsince a previous data copy to the target memory devices is to be copiedagain in a subsequent data copy to the target memory devices; when thestale data flag indicates that the unchanged data is to be copied, copyto the target memory devices the data files having the unchanged datafrom the data file; and when the stale data flag indicates that theunchanged data is no to be copied, skip copying to the target memorydevices of the data files having the unchanged data from the data file.7. The system of claim 1, wherein the at least one processor is furtherconfigured to: generate an executable database script defining there-arranging of the filegroups based on the priority index of the datatables associated with the data files; run the executable script toorder the data files in the copy queue.
 8. The system of claim 1,wherein the at least one processor is further configured to: obtaininformation relating to an ordering of a data file in the copy queue ina previous copy of the data files to the target memory devices; andorder the data files in the copy queue based on the obtainedinformation.
 9. A method for copying data from a source computingenvironment to a target computing environment, comprising: receiving acommand to copy a plurality of data files from one or more source memorydevices of the source computing environment to one or more target memorydevices of the target computing environment, wherein the source memorydevices store a database comprising the plurality of data files, whereineach of a plurality of data tables defined for the source computingenvironment includes data from one or more of the data files; detectingthat the one or more target memory devices have insufficient memoryspace to accommodate the plurality of data files; calculating a valuecoefficient for each of the plurality of data tables, wherein the valuecoefficient corresponding to each data table is indicative of a relativeimportance of the data table among the plurality of data tables, whereinthe value coefficient is determined for each data table based at leaston one or more interaction metrics related to the data table indicatingan amount of one or more types of interactions performed in relation tothe data table, an importance index assigned to the data table,user-defined numerical weights for each interaction metric and theimportance index, and a criticality index based on a relationship of thedata table with other data tables of the database; assigning a priorityindex to each data table based on the calculated value coefficient ofthe data table, wherein a higher priority is assigned to a data tablehaving a higher value coefficient; ordering the data files in a copyqueue for copying to the target memory devices based on the priorityindex of the data tables associated with the data files, wherein a datafile that includes data associated with a data table with a higherpriority index is ordered earlier in the copy queue; and schedulecopying of the ordered data files to the target memory devices accordingto the copy queue.
 10. The method of claim 9, wherein: the interactionmetrics related to each data table comprises one or more of a number ofread operations per megabyte (MB) in the data table and a number ofwrite operations per MB in the data table; and the importance indexassigned to each table comprises a Key Performance Indicator (KPI)index.
 11. The method of claim 10, wherein determining the criticalityindex for each data table comprises: determining that at least one otherdata table calculates a value in at least one data field based on datafrom the data table; and calculating the criticality index of the datatable based on the KPI index of the data table and the criticality indexof the at least one other data table, wherein the criticality index of adata table is higher when one or more other data tables depend on thedata table as compared to the criticality index of the data table whenno other data tables depend on the data table.
 12. The method of claim11, determining the value coefficient of the data table comprises:weighting each of the interaction metrics and the importance index basedon the corresponding user defined weights; and determining the valuecoefficient of the data table by adding the weighted interactionmetrics, the weighted importance index and the criticality index of thedata table.
 13. The method of claim 9, further comprising: obtaining adata date range defined for the data in the data table; and copying oneor more data files having data associated with the data table that iswithin the data date range.
 14. The method of claim 9, furthercomprising: obtaining a stale data movement flag for the data table,wherein the stale data movement flag indicates whether data from thedata table that has remain unchanged since a previous data copy to thetarget memory devices is to be copied again in a subsequent data copy tothe target memory devices; when the stale data flag indicates that theunchanged data is to be copied, copying to the target memory devices thedata files having the unchanged data from the data file; and when thestale data flag indicates that the unchanged data is no to be copied,skipping copying to the target memory devices of the data files havingthe unchanged data from the data file.
 15. A non-transitorycomputer-readable medium for copying data from a source computingenvironment to a target computing environment, the computer-readablemedium storing instructions that when executed by a processor cause theprocessor to: receive a command to copy a plurality of data files fromone or more source memory devices of the source computing environment toone or more target memory devices of the target computing environment,wherein the source memory devices store a database comprising theplurality of data files, wherein each of a plurality of data tablesdefined for the source computing environment includes data from one ormore of the data files; detect that the one or more target memorydevices have insufficient memory space to accommodate the plurality ofdata files; calculate a value coefficient for each of the plurality ofdata tables, wherein the value coefficient corresponding to each datatable is indicative of a relative importance of the data table among theplurality of data tables, wherein the value coefficient is determinedfor each data table based at least on one or more interaction metricsrelated to the data table indicating an amount of one or more types ofinteractions performed in relation to the data table, an importanceindex assigned to the data table, user-defined numerical weights foreach interaction metric and the importance index, and a criticalityindex based on a relationship of the data table with other data tablesof the database; assign a priority index to each data table based on thecalculated value coefficient of the data table, wherein a higherpriority is assigned to a data table having a higher value coefficient;order the data files in a copy queue for copying to the target memorydevices based on the priority index of the data tables associated withthe data files, wherein a data file that includes data associated with adata table with a higher priority index is ordered earlier in the copyqueue; and schedule copying of the ordered data files to the targetmemory devices according to the copy queue.
 16. The non-transitorycomputer-readable medium of claim 15, wherein: the interaction metricsrelated to each data table comprises one or more of a number of readoperations per megabyte (MB) in the data table and a number of writeoperations per MB in the data table; and the importance index assignedto each table comprises a Key Performance Indicator (KPI) index.
 17. Thenon-transitory computer-readable medium of claim 16, wherein determiningthe criticality index for each data table comprises: determining that atleast one other data table calculates a value in at least one data fieldbased on data from the data table; and calculating the criticality indexof the data table based on the KPI index of the data table and thecriticality index of the at least one other data table, wherein thecriticality index of a data table is higher when one or more other datatables depend on the data table as compared to the criticality index ofthe data table when no other data tables depend on the data table. 18.The non-transitory computer-readable medium of claim 17, whereindetermining the value coefficient of the data table comprises: weightingeach of the interaction metrics and the importance index based on thecorresponding user defined weights; and determining the valuecoefficient of the data table by adding the weighted interactionmetrics, the weighted importance index and the criticality index of thedata table.
 19. The non-transitory computer-readable medium of claim 15,wherein the instructions further cause the processor to: obtain a datadate range defined for the data in the data table; and copy one or moredata files having data associated with the data table that is within thedata date range.
 20. The non-transitory computer-readable medium ofclaim 15, wherein the instructions further cause the processor to:obtain a stale data movement flag for the data table, wherein the staledata movement flag indicates whether data from the data table that hasremain unchanged since a previous data copy to the target memory devicesis to be copied again in a subsequent data copy to the target memorydevices; when the stale data flag indicates that the unchanged data isto be copied, copy to the target memory devices the data files havingthe unchanged data from the data file; and when the stale data flagindicates that the unchanged data is no to be copied, skip copying tothe target memory devices of the data files having the unchanged datafrom the data file.